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C/3 | Abstract 



The emergence of dynamical structures in multi-agent systems is 
analysed. Three different mechanisms are identified, namely: (1) 
sensitive-dependence and convex coupling, (2) sensitive-dependence 
and extremal dynamics and (3) interaction through a collectively gen- 
erated field. The dynamical origin of the emergent structures is traced 
back either to a modification, by interaction, of the Lyapunov spec- 
trum or to multistable dynamics. 

PACS: 05.45.-a, 87.23.-n 
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g : 1 Introduction 

Organization and structure are ubiquitous in natural phenomena and much 
of the scientific endeavor is aimed at the discovery of patterns in the raw data 
supplied by Nature. Whenever a pattern is detected, it can be used to ob- 
tain a compressed description of the phenomenon or to predict its outcome. 
Prediction through compressed descriptions is also the way many living be- 
>• . ings deal with the external world, ants included 1J. If a natural process is 

stationary, the discovery of its patterns by an observer is of no consequence 
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for the system itself. It is merely an information processing feature of the ob- 
server. However, for evolving composite systems, the emergence of collective 
patterns may be a determining factor in the process of coevolution. 

To understand what a pattern (or structure) is, how it is represented and 
how it may be used for prediction, has been the subject of a great deal of 
research in the past. This ranges from time series prediction (for reviews 
see |2] |B|), to stochastic model identification |lj [Oj, to dynamical system 
reconstruction [7] [S] , to codingjU] [ID] and to the quantitative characteriza- 
tion of the complexity of patterns ^1] JT2] J3I- Of course, references here 
cannot be exhaustive nor make justice to a very large body of interesting 
work. They are only representative of the type of work developed in each 
approach. 

Concerning the question of what a structure is and how it may be used 
for prediction, a good part of the work done in the past fits in the general 
scheme of Crutchfield's computational mechanics^T^. A different question, 
however, concerns the dynamical mechanisms by which collective structures 
arise on composite systems. And also why their dynamical behavior may be 
so different from the dynamics of the components when in isolation. 

Cross and Hohenberg|15j discussed pattern formation in systems mod- 
elled by partial differential equations, by analyzing the instabilities of the 
homogeneous states. Near the instabilities the dynamics is described by am- 
plitude equations which characterize the collective variables. Here a different 
approach is followed. Systems composed of many agents in interaction, each 
one having simple dynamics, are considered. No collective variables are built 
in the model to begin with, and any collective structures that may be ob- 
served should appear as emergent properties of the dynamical process. 

What is meant by simple dynamics of the agents must however be clar- 
ified. The idea is that the dynamics is simple to describe in law, but not 
that it has simple orbits. In short, a dynamical law with small sophistica- 
tion[16 , but capable of generating orbits of high Kolmogorov complexity. A 
paradigmatic example is multiplication by p (mod.l) (p = 2, 3, ...). 

x n+1 =px n (mod.l) (1) 

It has an invariant measure absolutely continuous with respect to Lebesgue, 
positive Lyapunov exponents and Kolmogorov entropy, as well as orbits of 
all types. It is clear that, even if simple to describe, the dynamics of each 
agent must contain enough dynamical freedom for interesting behavior to be 



obtained when the agents are put in interaction. Otherwise, if dead rocks 
are added to a barren landscape, all one obtains is a rocky ground. 

In a multi-agent system, a temporal or spatial structure is defined as 
a phenomenon which has a time or space scale much larger than the cor- 
responding scales of the individual agent dynamics. Structures and other 
collective properties of multi-agent systems may be rigorously characterized 
by ergodic invariants|17j- [TTIj . A short summary of the main ergodic invari- 
ants that may be used to characterize multi-agent systems is included in 
the Appendix. When a unique invariant measure controls the dynamics, the 
ergodic invariants provide an adequate characterization of the system behav- 
ior. However in multistable systems, many different measures must be taken 
into account and a different type of parameters must be used. This is also 
sketched in the Appendix. 

In the same way as there is no unique way to characterize the com- 
plexity of dynamical systems, each feature requiring a different complexity 
parameter [TT] (THj, one should not expect to find a unique universal mech- 
anism responsible for all structure-generating effects. In this paper three 
different mechanisms are identified, of which some examples are studied, 
namely 

(1) Sensitive-dependence and convex coupling 

(2) Sensitive- dependence and extremal dynamics 

(3) Interaction through a collectively generated field (Multistability and 
evolution) 

In some of these mechanisms an important role is played by sensitive- 
dependence, that is, by the fact that the individual agent dynamics has 
positive Lyapunov exponents, as in Eq.(JI]). Without interaction the system 
would have a degenerate positive Lyapunov spectrum. The interaction lifts 
the degeneracy and it is the fact that different directions in phase-space 
acquire different separation dynamics that creates the collective dynamical 
structures. In particular when a Lyapunov exponent approaches zero from 
above, it creates a feature with a very long time scale. The modification 
of the Lyapunov spectrum arises either from varying interaction strength or 
from extremal dynamics, that is, the mechanism by which only the agent 
under the largest stress is allowed to evolve. 

A remarkable exception to the above paradigm of Lyapunov spectrum 
modification occurs when there are no direct interactions between the agents, 
which only react to a collective variable, that they themselves create. In this 
case, sensitive-dependence of the agent dynamics influences the fluctuations, 



but self-organization and collective variables are mainly controlled by evolu- 
tion processes and multistability of the dynamics. 

2 Sensitive-dependence and convex coupling 

Here the individual agent dynamics is assumed to have positive Lyapunov 
exponents. Without interaction, the system would have a degenerate spec- 
trum of positive Lyapunov exponents. Convex coupling, as in Eq.(j2J), has a 
contractive effect. Therefore, for sufficiently large interaction strength, some 
of the Lyapunov exponents approach zero from above. Physically, the mech- 
anism of convex coupling relates, for example, to a situation where there is a 
limitation on the range of options and influences that determine the actions 
of the agents. Then if an agent receives an influence from someone else there 
is a correspondent decrease of the influence of its own state in the future 
evolution. On the other hand the intensity of the interactions between the 
agents may depend on the history of past interactions or on the number of 
agents that occupy the same volume of space. 

This mechanism will now be illustrated by one example. It deals with 
a situation where the number of interacting agents varies, according to a 
reproduction and death scheme, and the strength of interaction depends 
on the number of agents at any given time. Of particular interest is the 
population control effect of the correlations. 

The model is a system of Bernoulli agents on a circle with nearest- 
neighbor interactions by convex coupling, namely 



x .(t + 1) = (1 - c)f{ Xi {t)) + °- (f(x i+1 (t)) + /(s<_i(*))) (2) 

with f(x) = 2x (mod. 1) and periodic boundary conditions. The agents 
are assumed to live in a limited space, the intensity of the coupling being a 
function of the total number N of agents, for example 
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(1 - e~ aN ) (3) 



With fixed coupling, this model was used in the past to illustrate the 
behavior of the ergodic invariants for self-organization [T^j and also in several 
other studies of the dynamics of coupled map lattices. 



Here the coupling becomes a dynamical variable as well, by a reproduction 
and death mechanism defined as follows: 

- After each R time cycles, the system is examined, agents which at that 
moment have Xi > 0.5 are coded 1 and those for which Xi < 0.5 are coded 0. 

- Then, configurations 0110 are candidates for reproduction with proba- 
bility p r and configurations 0000 are candidates for death with probability 

Pm- 

- Reproduction is the transition 0110 — > 0X110 with the state of the new 
agent X being chosen at random in the interval (0, 1). 

- Death is the transition 0000 -> 000. 

It is clear that without coupling the two configurations 0110 and 0000 
appear, on average, the same number of times and the variation of the pop- 
ulation density depends only on the relative values of p r and p m . With cou- 
pling the situation will be different and the model shows how correlations, 
generated by coupling, influence the inter-agent evolution mechanism. 

In this example a rigorous characterization is possible of the structures 
that develop through interaction. As explained in [19J this characterization is 
obtained through the computation of the Lyapunov exponents from which a 
structure index is constructed (see the Appendix). Further insight is obtained 
from conditional exponents as well, but they will not be used here. The 
Lyapunov exponents for the dynamical system in (J2j) are 

A fc = logJ2(l-c) + 2ccos(— fcj 1 (4) 

k — 0, • - • , N — 1. They are all positive for c < 0.5 and when the coupling 
varies above this value one observes the crossing through zero of each in- 
dividual Lyapunov exponent and successive changes in the structure of the 
system. That means that, for c ^ 0, each collective mode has a different 
probability, a collective mode being frozen each time a Lyapunov exponent 
reaches the zero value. 

The eigenvectors corresponding to each exponent are |e m6,fe } with 6k = 
?§k , k = 0, • • • , N - 1. Therefore 
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are the coordinates of the collective eigenmodes. In Fig.l one shows the 
average energy Ek = (yl) of the collective modes for several values of the 



coupling (for a system with 100 agents). In all plots the mode k — 0, that 
is just the average over all agents, is not shown. The expected suppression 
and freezing of many modes is quite apparent. Notice that some modes that 
are suppressed for certain values of the coupling become restored for higher 
values. This is apparent, for example, for c = 1. 

Fig. 2 shows the evolution of the population plotted against the reproduction- 
death cycle number. The probabilities are p r = 1 and p m = 0.5. This is 
a situation for which, without coupling, the population would grow indef- 
initely. However with the density- dependent coupling (jSJ) the population 
becomes controlled with fluctuations around some average value. Three of 
the plots show this stabilization starting from different initial conditions. 
Even if the population stabilizing dynamical mechanism leads to a non-zero 
average value, a large fluctuation may lead to extinction, as shown in the 
last plot. 

The population stabilizing mechanism is a consequence of the correlations 
that develop as a result of the unequal distribution of energy among the 
collective modes, caused by the coupling. Fig. 3 shows the relative probability 
of each one of the 16 different configurations of four neighbors {x\X2X^Xi), 
labelled by X\ + 2 x x 2 + 4 x x 2 + 8 x x 2 . As seen in the first plot, without 
coupling all configurations are equiprobable. 

Changes in the dynamical structure appear associated to the points where 
each Lyapunov exponent crosses zero. These are the points where the struc- 
ture index diverges because the time scale associated to the vanishing Lya- 
punov exponent becomes infinite. In this sense these points are similar to 
statistical mechanics transition points. This transition is even more dra- 
matic when a large number of Lyapunov exponents crosses zero simultane- 
ously. This is, for example, the case for the globally coupled model studied 
in Ref.^Zj. The transition points correspond to well defined (sharp) values 
of the parameters (coupling parameters, population density, etc.). A system 
may approach such points in the course of its life, by dynamical evolution of 
the population density, as seen above, or by environment changes. However, 
to generate a spontaneous approach to criticality, without fine tuning of the 
parameters, a different mechanism is required. As seen later on, extremal 
dynamics together with positive Lyapunov exponents of the individual dy- 
namics is such a mechanism that drives a system to the edge of criticality, 
that is, to the point where the Lyapunov exponents approach zero. 



3 Criticality and extremal dynamics 

Here one analyses the case where the individual dynamics is sensitive de- 
pendent, that is, it has at least one positive Lyapunov exponent, but the 
collective dynamics is of the extremal type. That means that at each time 
step only the agent under the largest stress is allowed to evolve. Depending 
on the specification of the dynamics the largest stress may be the largest 
driving force or the largest or smallest value of a state variable. For exam- 
ple in the Bak-Sneppen|2II] model the agent that evolves (together with its 
neighbors) is the one that has the smallest value of the state variable and 
in the train model[21J 22^ the agent that moves is the one that suffers the 
greatest driving force. 

The prescription of extremal dynamics is a feature that simulates friction 
or resistance to change in the dynamical system. When parallel dynamics is 
replaced by these type of extremal dynamics, a dramatic effect takes place 
in the spectrum of Lyapunov exponents, in the limit of a large number N 
of agents. For the computation of the Lyapunov exponents of the coupled 
system, instead of a tangent map matrix involving all the partial derivatives, 
one has now the product of matrices which have ones on the diagonal al- 
most everywhere and only one non-trivial r x r block, r being the number 
of neighbors that evolve at each time step. Therefore the Lyapunov expo- 
nents are obtained, on average by the root Jr of the r x r blocks. Therefore 
if the exponents of the r x r blocks are positive then, for large N, all the 
Lyapunov exponents approach zero from above, independently of any other 
characteristics of the dynamics. Hence sensitive-dependence of the individual 
dynamics plus extremal dynamics leads the system to the edge of criticality, 
in the sense that vanishing Lyapunov exponents means that there is no nat- 
ural time scale for the separation dynamics. Recall that these are the points 
where the structure index diverges (see the Appendix). 

Models of these type belong to the general class of self- organized criticality 
(SOC) [23] although not all SOC models that have been proposed display the 
above described mechanism in all its purity. At the end of the section a 
comment will be made about this. 

A great deal of work, on the SOC phenomenon, has been done by many 
authors. Here I only want to emphasize those features that relate to the 
Lyapunov spectrum. For definiteness I will concentrate on a continuous 
version of the Bak-Sneppen model^H] which is a C°°— dynamical system 
defined as follows: 



Let it G [0, 1)^ be the vector of coordinates of the agents and Ti(ltf ) the 
function 

j=i-\-nv 

Ul?)= J] IKl + e^-^y 1 (6) 

j=i-n v k^j 

For large a the function is nearly zero if i is in the riy— neighborhood of the 
agent with the minimum coordinate (x m m) and is nearly one otherwise. The 
dynamics of the model is defined by 

x t {t + 1) = Vi{ -&) Xi (t) + (1 - r 4 ( 1?)) / (xi(t)) (7) 

with f(x) = 2x (mod. 1). For ny — 1 and in the limit of large a this 
is equivalent to the original Bak-Sneppen model [20]. Notice that the usual 
operation of finding the agent with the smallest barrier is replaced here by 
an infinitely different iable operation and all the dynamical system techniques 
and results may be safely applied. 

For large a the Lyapunov exponents are 

A ~ log (2)* iV times (8) 

Therefore, as stated before, as N — > oo, A — > + . Hence, in this limit, there 
being no natural scale for the dynamics, exponential decay terms must dis- 
appear in all relaxation phenomena, leaving only the power law pre-factors. 

This dynamical system displays some unusual features, related in par- 
ticular to the nature of what has been improperly called its "attractor". 
In the N — > oo limit the one-agent probability density is uniform above 
0.67 and zero below this threshold. The marginal density for any finite 
number n of agents is the projection on a n— dimensional hyperplane of the 
iV— dimensional hypercube of side (1-0.67). This hypercube however is not 
an attractor because it carries zero or negligible (for finite N) measure. Also 
it is not a repeller because there are many neutral directions corresponding 
to the directions that are not neighbors to the minimum coordinate nor even 
a weak repeller because it is not an invariant set. I will call such a set a 
ghost weak repeller, sets of this type being characterized by the following 
conditions: 

(i) existence of repelling and neutral directions 

(ii) zero or negligible measure 



(iii) full measure on the projection to hyperplanes up to dimension N — k, 
N being the dimensionality of the system and k a finite number. 

Fig. 4 illustrates some of these properties for the model defined in Eq.flJJ) 
with iV = 100, riy = I and a = 1000. The figure shows the one- and two-agent 
marginal distributions, the distribution of the distances to the hypercube 
of side (1-0.67) and finally the scaling of the avalanches. The third plot 
emphasizes the negligible measure that is carried by the hypercube itself, to 
be compared with the structure of the marginal distributions. 

Whereas in the model discussed in the previous section, the creation of 
new structures, associated to the modification of the Lyapunov spectrum, 
occurred at particular values of the parameters, here no fine tuning of the 
parameters is needed, the system self-organizing itself spontaneously in a 
critical state. In the BS-model, as stated above, the establishment of the 
long-range temporal correlations is very naturally associated to the disap- 
pearance in the N — > oo limit of any characteristic time scale in the Lya- 
punov spectrum. Therefore the BS model is a very paradigmatic example 
of the SOC phenomenon in the sense that it always become critical in the 
N — ► oo limit, independently of any other details (value of riy, function / 
chosen for the dynamics of x m ; n , etc.). This may not be the case for all SOC 
models that have been proposed. 

For example, Zhang's model [2I| was studied as a dynamical system by 
Blanchard, Cessac and Kriiger[25j. In their formalism, Zhang's model is a 
dynamical system of skew-product type with the first factor corresponding 
to the activation of one site and the second to the energy relaxation process. 
Their unit of time is not the natural unit of time, instead each evolution 
step corresponds to an activation and a full avalanche. Therefore each step 
corresponds to different lengths of physical time, depending on the size of 
the avalanche. In their step units, there is one positive Lyapunov exponent 
(log AT) corresponding to the activation dynamics, the rest of the dynamics 
being contractive. Denote by t the average number of iteration steps (average 
duration of an avalanche) , by a the average number of distinct relaxing sites in 
one avalanche and by d the dimension of the lattice. Then, one estimates that 
the expanding Lyapunov exponent in physical time units is of order A log (N). 
If Aj is one of the contracting Lyapunov exponents, in physical units it is of 
order N Aj. Therefore all characteristic time scales will disappear only if 
t grows faster than log (N) when N —> oo and a grows slower than A" when 
A" — » oo. This means that in this model there may be ranges of parameters 
for which the SOC phenomenon is not observed. A similar conclusion is 



reached by the authors [2E] using other approach. 

In the discrete version of the train model proposed by Vieira[22j the 
activation dynamics, being the addition of a fixed quantity 5f to the first 
block, is neutral as far as the Jacobian is concerned. The only non-zero 
contribution to the Lyapunov comes from the derivative of the updating 
function 0' . It has been noticed by the author[221 that SOC behavior is only 
obtained if \<f>'\ > 1. With the estimate 4log|0'| for the global Lyapunov 
exponent, in the N — > oo limit, this case is seen to correspond, once again, 
to an approach to zero from above. 

4 Interaction through collective variables. Mul- 
tistability and evolution 

In all models studied before, there is some sort of direct interaction between 
the agents giving rise to the collective behavior. Another class of models is 
the one where the interaction is mediated by a collective variable. On the 
other hand, the collective variable is an aggregate result of the state variables 
and actions of the agents. Therefore the agents react to an aggregate variable 
that they themselves create. 

In most models of this type, in addition to the interaction through the 
collective variable, there is also an evolution mechanism which plays an im- 
portant role in organizing the system. Therefore the dynamics of the system 
is a composition of two dynamical laws. One is the (fast) dynamics of interac- 
tion through the collective variable, the other the (slow) evolution dynamics. 
Sometimes it turns out that the essential mechanism self-organizing the sys- 
tem is the evolution mechanism, a slow dynamics, whereas the fast dynamics 
only provides the multi-attractor background which is selected by the slow 
evolution. As an example of this mechanism two models will be studied. 

4.1 Coupled map minority model 

Inspired on Brian Arthur's bar modelj2E], models have been proposed|27] 28, 
[29] where the agents choose the value of a variable (±1, for example) and 
those that are on the minority group win a point. In a continuous version [T9]. 
which is qualitatively equivalent to the discrete one, a fixed number c between 
zero and one is chosen, which one calls the cut. The cut divides the interval 
[0, 1] into two parts. Then each agent chooses a value x% between and 1, 
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and the average x m = -^ J2i x i * s computed. The winning agents are those 
for which Xi lie on the side opposite to x m . That is, the payoff of agent i at 
time t is 

Pi(t) = 2^~ si S n {(^W - c ) (^(*) - c )» ( 9 ) 

At each time t the dynamics of agent i is a function of the average value x m 
at time t and of a parameter 014 that characterizes his strategy 

Xi{t + l) = fi{x m {t),oti) (10) 

Here one considers for the function fa either a shifted tent map 

/i(x) = 2 + 2xsign Q - (ar + at) J (modi) (11) 

or a shifted p-ary multiplication 

/j(z) =?(& + «.) (modi) (12) 

«i being a number between zero and one, a different one for each agent. 

At first the strategies, that is the a;'s, are randomly chosen. Then each 
r time steps, k agents have their strategies modified. The k! agents with less 
earnings in that period have new a's chosen at random and the remaining 
k — k' copy the a's of the k — k' best performers with a small error. This 
is the evolution dynamics of this model, the fast dynamics being the one 
in Eq. (|10j) . The variables of the full dynamical systems are (xj, a*;), these 
variables being coupled by the interplay of fast and evolution dynamics. 

In the discrete minority models originally proposed, each agent has several 
strategies at his disposal and at each time step he chooses the strategy with 
the best virtual record. The periodic replacement of the worst strategies by 
the best ones, used here, is qualitatively equivalent and, in addition, provides 
a clear separation between the two types of dynamical laws that operate in 
the model. In particular by changing the ratio - one may explore different 
time scales for the (fast) agent dynamics and the (slow) renewal and copy 
dynamics. This clear separation between the two dynamics is important be- 
cause, as it will be seen, they play very different roles in the self-organization 
of the system. 

The most interesting feature of the system dynamics is the fact that after 
a certain time it approaches a regime where the average value x m oscillates 
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around the value of the cut c, even when c is very different from the random 
value 0.5. Fig.5 shows the typical behavior of the system for c = 0.7, the 
map f\ being the tent map. The two upper plots show the approach of x m 
to the self-organized steady-state through several steps corresponding to the 
evolution cycles and the fluctuations of x m around the cut. The number of 
agents is N = 100, k = r = 10 and k = 3. The two lower plots in the figure 
show the distributions of x m and of the fraction of winning agents 

P = ^£P, (13) 

i 

The average value and standard deviation of x m are x~^ = 0.694 and a(x m ) = 
0.02, to be compared with x~^ = 0.5 and cr(x m ) = 0.288 that would be 
obtained for a uniform random choice of values between and 1 for the agent 
variables. The fact that x m is close to the cut maximizes the percentage of 
winning agents, which for the data in Fig.5 is P = 0.488 with o{P) = 0.132. 

The organization of the system's collective variable around the cut c is 
easy to understand. Suppose that at a certain time x m < c. Then, the 
evolution dynamics tends to copy the strategies of the agents that on aver- 
age have Xi > c. This drives the average x m to higher values, closer to c. 
Conversely if x m > c the effect is just the opposite one. Hence x m tends to 
oscillate around c. For a minority model with agents having several strate- 
gies at their disposal, the choice of the strategies with the best virtual record 
has the same effect. Cavagna30j pointed out the irrelevance of the memory 
size (the number of past time steps that the agents use in their strategies) 
and stated that the important issue is that all agents use the same collective 
information. A more accurate statement would be that the organization of 
the collective variable around c depends only on the evolution dynamics, not 
on the details of the (fast) dynamics of the agents. 

However, the dynamics of the agents, that is, the nature of their strategies, 
has an effect on the type of fluctuations around c. The Lyapunov exponents 
for the dynamics of x m and for the dynamics of the agents characterize these 
fluctuations. The dynamics of x m is 

X m (t+1) = — J2 fi( X m (t)+Oi) (14) 



12 



the Lyapunov exponent being 



1 / 1 

A = lim — log — - 

fc-oc k \ N 



^2 fi(Xm(t) + &i) 



1 

N 



^2 fi( x m(t + k) + ai 



(15) 



For the tent map A, assuming, for a large number agents, a uniform distribu- 
tion of the a's over the interval, -^ |£V f[\ is of order -4=, hence A is negative 

of order — | log N. For the p-ary map A = p. 

For the dynamics of the agents the Jacobian matrix is 



I NJl NJ\ 



DT 



NJl \ 



(16) 



V nJn nJn ' ' ' nJn / 



The eigenvalues of (DT fe ) (DT fe ) are N — 1 zeros and one equal to 



(17) 



Therefore there is only one non-trivial Lyapunov exponent identical to the 
Lyapunov exponent of the x m dynamics. 

It is the evolution dynamics that organizes the system, driving x m towards 
the cut. The fast dynamics controls the nature of the fluctuations around 
this value. For the tent map, all the Lyapunov exponents being negative, in 
the time intervals of duration r between the evolution steps the dynamics 
settles down at a fast rate to a fixed point or periodic orbit. Nevertheless the 
behavior of the collective variable around its average value, as shown in Fig. 5 
is quite irregular. The reason why this is compatible with the fast contraction 
associated to negative Lyapunov exponents is the sensitivity of the attractor 
to small changes on the agents strategies, that is, to the variables where 
the evolution dynamics acts. Therefore the (xj, a^) —dynamical system is a 
multi-attractor system. For each fixed set of strategies and initial conditions 
the system converges rapidly to a period orbit. However small changes of 
the a— variables, induced by the evolution process, change the attractor to 
which the system converges. 
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For the p-axy maps the existence of one positive Lyapunov exponent 
changes the nature of the fluctuations around the mean collective value, 
meaning that in this case the system is still a multi-attractor one, but the 
attractors are not necessarily periodic. This is shown in Fig. 6 where the 
same quantities as in Fig. 5 are plotted. The fluctuations now partly spoil 
the self-organization induced by evolution. The average values and standard 
deviations for the data in Fig. 6 are x^ = 0.554, a(x m ) = 0.145, P = 0.378 
and cr(P) = 0.223. Also in this case, the evolution dynamics controls the 
collective behavior and acts as a selector of the attractors of the fast agent 
dynamics. The difference to the previous case is that here, rather than peri- 
odic orbits, one has non-periodic attractors. 

In contrast to the mechanisms discussed before, where the approach 
towards + of the positive Lyapunov exponents is the source of the self- 
organized collective variables, here it is a multi-attractor evolution mecha- 
nism that determines the emergence of such variables and, on the contrary, 
positive Lyapunov exponents may, to some extent, spoil the self-organization 
of the system. 

4.2 A market- like game 

In the minority model, as we have seen, the primary interaction between the 
agents takes place through an external collective variable, which they them- 
selves create. In addition there is another dynamical (slower) mechanism 
which is the copy of the best strategies by the worst players or, alternatively 
the choice by each player of the best virtual strategy among a number of 
strategies put at their disposal. This paradigm is very much alike what hap- 
pens in a market, where, among other things, investors react to the stock 
prices, which they themselves influence through their investments. At the 
same time they evolve in time trying to adjust their strategies in order to 
maximize their profits. Therefore, once again, we have an interaction through 
a collective variable and an evolution mechanism driven by the desire to max- 
imize a cost function (the profit). From the lessons learned in the minority 
model, one is led to expect the collective variables to be controlled by the 
evolution mechanism with the agent dynamics providing the attractor back- 
ground. 

Many factors play a role in a real market. Here no attempt is made to 
take into account all the relevant factors, nor to build an accurate model of 
the market place. The objective is to isolate some of the mechanisms that 
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presumably play a role in the market and, by stripping the model from other 
(inessential?) complications, to exhibit and understand the purified effect of 
these factors. In a real market, many factors, endogenous and exogenous, 
play a role and one should not expect to find such a clear cause-effect re- 
lationship between dynamical laws and actual behavior. Nevertheless, as in 
other branches of science, the splitting apart of the dynamical components 
of a phenomena, may improve its understanding J3I] . 

We consider a set of investors playing against the market, that is they 
have some effect on an existing market that is influenced by other factors 
(other investors and general economic effects). This assumption implies that 
in addition to the impact function of this group of investors on the market, 
the rest of the impact is represented by a stochastic process. Therefore 

Zt+i = f{zt,w t )+r]t (18) 

represents the change in the log price (z t = log pt) with wt being the total 
investment made by the group of traders and rj t the stochastic process that 
represents all the other effects. 

In addition, no conservation law is assumed for the total amount of stock 
s and cash m detained by the group of traders. If p t is the price of the traded 
asset at time t, the purpose of the group of investors is to have an increase, 
as large as possible, of the total wealth m t +pt X s t at the expense of the rest 
of the market. 

For purposes of comparison with the minority model, here the collective 
variable z plays the role of the average value x m and the difference between 
the initial wealth and the wealth at time t 

^ = E H° + V* x s ?) - E H } + po x 4°) (19) 

i i 

plays the role of the total payoff P 

The market impact function 

Let p be the price of some asset, z = log(p) and io t the total sum of the 
buying and selling orders (in money units) for the asset. Buying orders are 
positive and selling ones negative. An important factor in the models is the 
effect of the magnitude of these orders on the price change of the asset, the 
so called market impact junction. Let small orders have an impact according 
to the loglinear law [7] [331 

OJt 

zt+i ~ z t = — +r] t (20) 
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The constant A, called the liquidity, controls the volatility of the market. It 
corresponds naturally to a first order expansion and satisfies the condition 

p(p(po,u;«),u;®) =p(Po,^ (1) W 2) ) (21) 

which one expects to be valid for small orders. However, as pointed out 
by Zhang [3*lj there is experimental evidence that this is not an accurate 
representation for large orders. Therefore a slightly different market impact 
function will be used. The reasoning used to motivate it has some relation 
to Zhang's although the result is somewhat different. 

When using Eq. (|20|) in a discrete dynamics model we are somehow ne- 
glecting the fact that the market takes different times to fulfill (and to react 
to) small and large orders. Therefore this should be taken into account when 
reducing the dynamics to a sequence of equal time steps. In particular the 
reaction of the market may be parametrized by a change in the A coefficient, 
which being related in first approximation to a random walk may vary by a 
factor proportional to \ft. Taking the time t to fill an order to be proportional 
to its size, one obtains 

zt+i ~zt= "\ 1/2 + Vt (22) 

A + Ai \u t \ 

For small orders one recovers the loglinear approximation and for very large 
orders Zhang's square root law. 

The agent strategies 

In first-order, two main types of informations are taken into account by 
the investors, namely the difference between price and perceived actual value 
(the misprice) 

zv t - z t = hg(v t ) - log(pt) (23) 

and the variation in time of the price (the price trend) 

z t - z t -i = log(p t ) - log(pt_i) (24) 

One may also consider differences of prices going further back in time. How- 
ever, the qualitative effect on the dynamics is basically the same and, in line 
with the main aim of isolating the fundamental constituents of the process, 
only these two pieces of informations will be considered. Consider now a 
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non- decreasing function f(x) such that /(— oo) 
useful examples are 



fi{x) 

Hx) 



9(x) 



i 



l+cxp(— 0x) 



and /(oo) = 1. Two 



(25) 



The information about misprice and price trend is coded on a four-component 
vector 7 



It 



I f(zv t - Zt)f(zt - Zt-i) \ 

f(zv t - Zt) (1 - f(z t - Z t -i)) 
(1 - f(zv t - Z t )) f(z t - Zt-i) 

V (1 - f(zv t - Zt)) (1 - f(z t - Z t -,)) J 



(26) 



The strategy of each investor is also a four-component vector a;W with entries 
— 1, 0, or 1. —1 means to sell, 1 means to buy and means to do nothing. 
Hence, at each time, the investment of agent i is a^ • 7 . A fundamental 
(value-investing strategy) that buys when the price is smaller than the value 
and sells otherwise would be a® = (1, 1, — 1, —1) and a pure trend-following 
(technical trading) strategy would be a^ = (1, — 1, 1, —1) . In this setting 
the total number of possible strategies is 3 4 = 81. For future reference the 
strategies will be labelled by a number 



n 



CO 



3 

fc=0 



n 



(0 



(27) 



Therefore the fundamental strategy is strategy no. 72 and the pure trend- 
following one is no. 60. 

As compared with a realistic market model, an important ingredient that 
is missing is to take into account transaction costs. In actual practice however 
excessive transaction costs are avoided by the introduction of thresholds in 
the agent strategies. Therefore neglecting both thresholds and transaction 
costs has a compensating effect and, qualitatively, the behavior is expected 
to be the same. 

Another factor that is sometimes considered in market models is the de- 
pendence of the strategies on the values of the prices not only at t and t — 1 
but also on a larger set of past times. However, if the lesson that is learned 
from minority models also applies here, the size of the memory and the details 
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of the agents strategies are not very important as far as the collective proper- 
ties of the model are concerned. What seemed to be important there was the 
copy mechanism in the evolution dynamics and the value of the Lyapunov 
exponents to control the fluctuations around (and away) from the value of 
the collective variable, the latter being mainly controlled by the evolution 
dynamics. 

The evolution dynamics that is considered here for the market model is 
similar to the one of the minority model. After a number r of time steps, s 
agents copy the strategy of the s best performers and, at the same time, have 
some probability to mutate that strategy. This evolution aims at attaining 
the goal of improving gains, while at the same time allowing for some renewal 
of the strategies. The percentage of each strategy changes in time and one 
may find whether some of them become dominating or stable and when this 
may occur. 

Figs. 7 to 10 show the results of some simulations of the model. The 
parameters that were kept fixed are r = 50, s = 10, Ao = 10000. The sim- 
ulations differ by the choice of the initial conditions and the existence or 
non-existence of evolution. For Fig. 7 an initial condition is chosen with all 
traders in the fundamental strategy and evolution is activated. It is seen 
that on average the price follows value, although its fluctuations are ampli- 
fied. In the last plot of Fig. 7 one shows the time evolution of the strategies 
distribution coded according to (|27|) . The fundamental strategy is seen to 
be stable, in the sense that it becomes dominant, not being invaded by any 
other of the strategies that are created by the mutation process. There are 
however a few other strategies that, after being created, survive the selec- 
tion process. This is true for example for the strategies 45 = (0, 1, — 1, —1), 
18 = (-1,1,-1,-1), 73 = (1,1,-1,0) and 75 = (1,1,0,-1). These sur- 
viving strategies are however similar to the fundamental one. When there 
is dominance of the fundamental strategies, the price increments dp have a 
Gaussian distribution. On the other hand the collective objective of increas- 
ing gains A t (Eq. (JT5j) ) * s achieved, as shown in the third plot of Fig. 7. 

For the simulation of Fig. 8 the initial condition contains 50% of funda- 
mental strategies (no. 72) and 50% of trend- following ones (no. 60). It is 
seen that the trend following strategies do not survive the selection process 
and are eliminated, after a transient period. The statistically stable situation 
that is obtained is similar to the one shown in Fig. 7. However the fundamen- 
tal strategy ceases to be stable if it occurs in the initial condition in smaller 
amounts (< 40%). The dependence on the initial condition is manifest in 



Fig. 9, where one starts for a completely random mixture of strategies in the 
initial condition. In this case, although the selection mechanism is still favor- 
ing at each evaluation cycle the best performers, the system never organizes 
itself to make A t grow. 

Finally the simulation in Fig. 10 is performed without evolution, with a 
fixed 50% of fundamental strategies (no. 72) and 50% of trend-following ones 
(no. 60). One sees in this case a large number of bubbles and crashes in the 
price evolution and the price increments distribution has fat tails. The last 
plot in Fig. 10 is an expanded plot of the bubble around time step 39800. 

To understand the nature of the dynamics that leads to the results of the 
simulations is useful to compute the Lyapunov exponents for the log-price 
(zt) dynamics. The Jacobian for the dynamics 



Z t -1 ) \ Zt 



IS 



(28) 



1 + 



M t =\ 92 *Ao+Ai|E^ W | az *-i Ao+AilEiU^I (29) 

V 1 o J 

the Lyapunov spectrum being obtained from 

lim \Mj +N _ l ---M?M t ---M t+N „ 1 \ l/2N (30) 

N— >oo 

Lyapunov exponents were computed for / = f 2 (Eq.([25|)) for several values 
of j3 and a 50 — 50 admixture of fundamental and trend-following strategies. 
Typically, that is for a very large range of /3, one obtains one Lyapunov 
number equal to zero and the other smaller but close to one. Therefore, 
typically, one has two negative Lyapunov exponents, one of them close to 
zero. 

Although more complex than the minority model, the similarities are 
very clear. In the minority model there is an objective variable (the payoff 
P) which drives the evolution and a collective variable x m to which the 
agents react in the short run. Here the objective variable that controls the 
evolution is p x s + m and the price is the collective variable. In both cases the 
average behavior of the collective variable is controlled by evolution and the 
fluctuations by the (fast) agent dynamics. Whether the objective variable 
reaches a stable behavior depends of course on the interaction between the 
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two dynamical laws. Multistability of the coupled dynamics, rather than the 
Lyapunov spectrum, seems in both cases to be the main structure-generating 
mechanism. 

5 Appendix. Some parameters characteriz- 
ing the dynamics of multi-agent systems 

In this appendix one collects the definitions and some properties of a few 
parameters which may be used to characterize in a quantitative manner the 
self-organization of multi-agent systems. Two different cases are considered. 
The first concerns systems where only one invariant measure controls the 
dynamics and the second is the case where many different measures come 
into play. 

5.1 Ergodic invariants 

Let a dynamical system evolve on the support of some measure \i which is 
left invariant by the dynamics. An ergodic invariant is a dynamical charac- 
terization of this measure 

i T 
I(p)= lim- J2 r(f n x ) (31) 






for xq /i— almost everywhere. 



5.1.1 Lyapunov and conditional exponents 

Let / : M —>■ M , with M C R m , \i a measure invariant under / and a 
splitting of M induced by S = R k x R m ~ k . The conditional exponents^ j 
[T7j are the eigenvalues Q and Q m of the limits 

lim (D k r(x)D k f n (x))^ (32) 

n^oo 

lim (D m _ k r*{x)D k r{x))& 

where Dkf n and D m _kf n are the k x k and m — kxm — k diagonal blocks of 
the full Jacobian. For k = m , Q m = Aj are the Lyapunov exponents. Both 
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the Lyapunov and the conditional exponents are ergodic invariants, with 
existence /x-almost everywhere guaranteed by the conditions of Oseledec's 
multiplicative ergodic theorem, in particular the integrability condition 

/ n(dx) log + ||T(x)|| < oo (33) 

T being either the Jacobian or its k x k and m — k x m — k diagonal blocks. 
The set of regular points is Borel of full measure and 

lim -log\\D k f n (x)u\\=^ k) (34) 

with O^uG E x /E x +1 , E x being the subspace of R k spanned by eigenstates 
corresponding to eigenvalues < exp(£J ). 

5.1.2 Dynamical selforganization 

Self-organization in a system concerns the dynamical relation of the whole 
to its parts. The conditional Lyapunov exponents, being quantities that 
separate the intrinsic dynamics of each component from the influence of the 
other parts in the system, provide a measure of dynamical selforganization 

N 

I(S, S, n) = y^ {hkifi) + h m - k (n) - h(n)} (35) 

fc=i 

the sum being over all relevant partitions R k x R m ~ k and 



hk(v) - Z^« >0 £i ; h m _ k (fi) - Z^f( m - fc ) >0 ^ m 5 h(jx) - J2\i>o^i 



5.1.3 Conditional entropies 

Consider cylindrical partitions adapted to the splitting R k x R m ~ k , 

„(fc) _ //7W /7W ...1 

" i '? 'r I ^ (36) 



-2 



where C- and C t are k and m — k -dimensional cylinder sets in R" 
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Let now ( be a generator partition for the dynamics {/,//}. The condi- 
tional entropies associated to the splitting R k x R m ~ k are 

^ = sup lim ^H (C V f~ l ( V • • • V /-"C I V {k) ) 



V W ri^co 



fj(m-k) = sup ii m _L.# (cv/ _1 cv---v/- n c |?/ m - fe )) ( ' J) ' ) 

„(m-fc) n-»oo n+1 

if (x | 77) being 

H ( X I 77) = -J J2 » { C ^ I *) ln ^ (^ (X) I ^ ^ ( 38 ) 

That is, the conditional entropies are the supremum over all cylinder parti- 
tions of the sum of the conditional Kolmogorov-Sinai entropies. 

5.1.4 The structure index 

A structure (in a collective system) is a phenomenon with a characteristic 
scale very different from the scale of the component units in the system. A 
structure in space is a feature at a length scale larger than the characteristic 
size of the components and a structure in time is a phenomenon with a time 
scale larger than the cycle time of the individual components. A (temporal) 
structure index may then be defined by 

where N is the total number of components (agents) in the coupled system, 
N s is the number of structures, Tj is the characteristic time of structure % 
and T is the cycle time of the isolated components (or, alternatively the 
characteristic time of the fastest structure). A similar definition applies for 
a spatial structure index, by replacing characteristic times by characteristic 
lengths. 

Structures are collective motions of the system. Therefore their charac- 
teristic times are the characteristic times of the separation dynamics, that 
is, the inverse of the positive Lyapunov exponents. Hence, for the temporal 
structure index, one may write 



s = ^E(£-i) 



22 



the sum being over the positive Lyapunov exponents Aj. Ao is the largest 
Lyapunov exponent of an isolated component or some other reference value. 

The temporal structure index diverges whenever a Lyapunov exponent 
approaches zero. Therefore the structure index diverges at the points where 
long time correlations develop. 

More details on the construction of the ergodic invariants and their in- 
terpretation as relevant properties of multi-agent systems may be found in 

D3I- 

5.2 Multistability parameters 

If a dynamical system has multiple attractors for the same set of parameters, 
then each attractor will have its own invariant measure. However in this 
case these measures are not of great practical interest. Instead, a global 
(Lebesgue) measure \i is defined in phase space, with respect to which the 
probability to be in the basin of attraction of each one of the attractors is 
computed. Several parameters may be used to characterize the multistable 
system. 

One is the number ua{N) of distinct attractors as a function of the num- 
ber N of degrees of freedom (number of agents) of the system. Alternatively 
one may define the scaling function for the number of attractors g n (N) such 
that 

lim — - — - = constant (41) 

Jv-oo g n {N) 

The diversity of possible dynamical behaviors when the initial conditions 
are chosen at random is characterized by the attractor entropy 

S(JV) = X>(fc)logM(&i) ( 42 ) 

bi being the basin of attraction corresponding to the i attractor. As in (|41|) 
a scaling function gA{N) may be defined for the entropy. 

When a multistable system is perturbed, by noise or by fluctuations in 
the parameters, migration between attractors takes place which, in addition 
to the intensity of the perturbation, is strongly influenced by the stability of 
the attractors and by the nature of the boundaries of the basins of attraction. 
Given a metric in phase space, the stability of the attractors may be char- 
acterized by its average strength s defined as the average of the minimum 
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distances d m [ n (i) between the attractors and the boundary of their basins of 
attraction, scaled by average size of a basin of attraction 

1 \-^ 

d min (i) (43) 



n A (N) 



l-i ^ 



d being the geometrical dimension of the phase space. 

Another important factor controlling attractor migration is the Hausdorff 
dimension of the basin boundaries. If this dimension is high (in some cases 
it may approach d) the noise-perturbed system may spend most of the time 
in such a riddled boundary, without ever setting in any particular attractor. 
This situation leads to a high degree of unpredicability, even higher than the 
usual chaotic (positive Lyapunov exponent) regime. 

A easier to measure characterization of the effect of attractor strength 
and basin boundary structure on the migration dynamics is the mean first- 
passage time r(e) between attractors as a function of the noise intensity e. 
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